CrashKill
Contents
General Agenda
Crashkill team works to track and escalate stability issues for both Desktop and Mobile products. Our activities include....
- Track general stability for all our releases and development channels.
- Triage Top Crashes to find owners, set priorities, and identify vender outreach issues.
- Ensure that crash bugs are logged and updated with new information.
- Identify top tools issues so we can find & fix faster.
- Identify the first thing we want to do to increase developer awareness and commitment to fixing top crashers.
- Identify the first thing we want to automate.
- Working with the Socorro team to get features on the roadmap that help us find and diagnose crash bugs better.
- Set topcrash threshold criteria.
Top Crashes
There are a number of reports that we triage regularly in order to stay on top of issues, most notably the topcrash reports as linked in the front page of Socorro crash-stats (see top links in the Reports & Tools section of this page). Finding reproducible steps and helping us make them actionable will have a direct and in many cases significant impact on our users due to the volume involved. Not all bugs are fixes in our code. Many are 3rd party issues that can be resolved through outreach and/or blocklisting.
Reports & Tools
Mozilla collects crash data which can be easily viewed using Socorro (a web interface to the crash reports). We use Socorro and its numerous reports to monitor stability rates on a daily basis.
- https://crash-stats.mozilla.com/products/Firefox (Firefox Desktop)
- https://crash-stats.mozilla.com/products/FennecAndroid (Firefox for Android)
- https://crash-stats.mozilla.com/products/B2G (Firefox OS)
The "Are We Stable Yet?" Stability Dashboard tries to collect all important numbers in one place, with colored indicators for hinting at what they mean.
We also have a list of additional reports that we have developed to help us understand certain crashes and trends. We plan over time to integrate these into Socorro and most of these are on the roadmap already.
- https://crash-analysis.mozilla.com/chofmann/
- https://crash-analysis.mozilla.com/rkaiser/ (see Overview for always-current links)
- Explosiveness, Weekly bug stats, FxOS crashes by B2G version, Flash topcrashes by version, and other specialty reports
- https://crash-analysis.mozilla.com/bsmedberg/
- https://crash-analysis.mozilla.com/bsmedberg/flash-summary.html (summary breakdown of crashes and hangs by Flash version)
- CrashKill/Flash Breakdown Data
- Reporting Malware (external link): https://www.techsupportalert.com/content/how-report-malware-or-false-positives-multiple-antivirus-vendors.htm
Planning
- CrashKill/Plan
- We are actively working with the Socorro team on a list of priorities for getting features into Socorro that will help crash analysis. We update the list quarterly.
Work Weeks
- Stability Work Week 2012 (Notes)
- Stability Week 2013 (Notes)
Weekly Crashkill Meetings
Up to the end of 2014, weekly meetings were held to talk about stability issues across releases and channels. Those now have been replaced by mailing list communication and ad-hoc meetings before and after releases.
